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EXECUTIVE  SUMMARY 


In  1983,  the  Air  Force  Military  Personnel  Center  Surgeon  General  (AFMFC/SG)  requested  a 
review  of  their  selection  process  for  awarding  medical  school  scholarships  through  the  Armed 
Forces  Health  Professions  Scholarship  Program  (AFHPSP).  The  purpose  of  the  review  was  to 
investigate  the  selection  procedures  used,  make  recommendations,  and  develop  decision  aids  to 
improve  the  selection  process.  A  review  of  the  selection  procedures  showed  that  fewer  applicants 
were  awarded  a  scholarship  as  the  selection  cycle  progressed,  due  to  a  rise  in  the  qualification 
standards  for  the  scholarship.  This  meant  that  people  were  being  rejected  late  in  the  selection 
cycle  who  might  be  more  qualified  than  those  given  a  scholarship  earlier. 

To  help  the  AFHPSP  selection  board  maintain  a  consistent  selection  policy  and  to  provide  for 
greater  discrimlnabillty  among  candidates,  a  policy  model  was  developed  with  aid  from  AFHPSP 
selection  board  members.  The  policy  model  was  designed  not  to  duplicate  the  selection  board  but 
to  provide  the  board  with  an  alternate  view  of  each  applicant.  To  gather  information  for  use  in 
the  policy  model,  a  questlonnal  re  was  also  developed  to  be  administered  by  the  Air  Force 
Recruiting  Service. 

Using  data  on  FY87  AFHPSP  applicants,  two  different  rank -orderings  were  created,  one  using 
the  selection  board  scores  and  the  other  using  the  policy  model  scores.  A  comparison  of  the 
rank-orderings  showed  that  there  was  60?  agreement  between  the  board  and  the  policy  model 
select /non -select  decisions.  Regression  analysis  on  the  difference  between  the  two  scores  showed 
that  the  difference  could  be  attributed  to  the  information  on  the  individual  used  by  the  policy 
model  and  not  by  the  board  and  a  greater  weight  given  to  Grade  Point  Average  by  the  AFHPSP 
selection  board.  The  report  concludes  with  a  list  of  possible  uses  of  policy  model 
rank -orderings  and  a  recommendation  to  establish  a  long-range  data  base  in  order  to  analyze  the 
selection  process  more  effectively.  AFWPC/SG  has  decided  to  use  the  Tie-Breaker  approach  to  use 
the  results  from  the  policy  model  starting  in  the  FY88  selection  cycle. 
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PREFACE 


This  work  was  performed  In  response  to  Request  for  Personnel  Research  (RPR)  83-24 
entitled  “Improved  Selection  Procedures  for  Air  Force  Physicians,"  submitted  by  the  Air 
Force  Military  Personnel  Center  Surgeon  General's  Office.  By  Improving  the  selection 
procedures  for  the  award  of  the  Armed  Forces  Health  Professions  Scholarship  Program,  the 
Air  Force  is  incre?«ing  the  quality  level  of  the  Air  Force  medical  service  and  the 
propensity  of  quality  physicians  to  remain  in  service.  The  auth?v-  thanks  Maj  Peter 
Vroom  (AFMPC/SG)  for  his  assistance  in  implementing  the  policy  model,  SSgt  Wayne  Getson 
(AFMPC/DPY)  for  his  programing  assistance.  Dr.  William  Alley  (AFHRL/MOT)  for  his  advice 
on  validation,  ILt  Daniel  Gerrig  (AFHRL/MOM)  for  his  work  in  laying  the  foundation  of 
this  study,  and  Hr.  Larry  looper  (AFHRL/MOM)  for  his  review  of  the  report. 
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AN  AIR  FORCE  SELECTION  MODEL  FOR  THE  ARMED  FORCES 
HEALTH  PROFESSIONS  SCHOLARSHIP  PROGRAM 


I.  INTRODUCTION 

Personnel  selection  systems  in  organizations  are  generally  managed  by  personnel  specialists, 
with  input  from  the  functional  managers  for  whom  the  newly  accessed  people  will  work.  For 
adnittance  to  educational  programs  that  are  funded  by  scholarships,  often  a  selection  committee 
composed  of  personnel  specialists  is  tasked  with  the  job  of  selecting  scholarship  winners.  The 
committee  selects  individuals  by  evaluating  the  applicants'  various  attributes.  Incorporated  in 
this  evaluation  is  the  expectation  of  successful  completion  of  training  as  well  as  success  on  the 
job.  Analysis  of  such  selection  systems  usually  focuses  on  determining  short-term  performance  or 
success  as  a  function  of  easily  measured  input  variables,  and  ignoring  more  abstract  factors  that 
may  Influence  selection.  The  analysis  technique  discussed  in  this  report  focuses  on  specifying 
some  of  the  abstract  factors  for  use  in  a  computer-based  selection  model. 

The  present  effort  addresses  the  manner  in  which  the  Air  Force  manages  the  selection  of 
candidates  for  the  Armed  Forces  Health  Professions  Scholarship  Program  (AFHPSP)  and  presents  a 
methodology  that  establishes  a  ranking  of  AFHPSP  applicants  to  aid  the  selection  process.  The 
lag  between  the  time  of  selection  and  the  time  that  job  performance  measures  can  be  taken  makes 
the  identification  of  those  AFHPSP  scholarship  applicants  who  will  be  successful  as  a  career  Air 
Force  officer-physician  a  challenging  task  for  Air  Force  decision  makers. 

The  history  of  AFHPSP  began  in  1972,  when  Congress  established  the  program  in  anticipation  of 
the  end  of  conscription.  Through  1981,  AFHPSP  was  in  direct  competition  with  the  National  Health 
Service  Corps  (NHSC)  Scholarship  Program,  which  was  designed  to  place  physicians  in  underserved 
areas.  The  increase  in  funding  for  AFHPSP  in  1980  and  the  discontinuance  of  NHSC  in  1981  gave 
the  Air  Force  a  favorable  appl icant-to-schol arship  ratio  (Hosek,  1983).  For  example,  in  1983  and 
1984,  the  Air  Force  had  approximately  three  applicants  for  each  available  scholarship.  This 
beneficial  selection  ratio  created  the  opportunity  to  examine  current  selection  procedures,  to 
develop  policies  that  capitalize  on  the  favorable  selection  ratio,  and  to  incorporate  policy 
changes  into  a  model  that  aids  the  selection  process. 


II.  PURPOSE 

In  selecting  AFHPSP  scholarship  winners,  the  Air  Force  has  the  following  goals:  (a)  to 

maximize  the  expected  length  of  service  from  persons  in  the  Medical  Corps  (physicians  in  the  Air 
Force),  (b)  to  access  high  quality  individuals  for  officers  in  the  Medical  Corps,  and  (c)  to 
ensure  the  continued  high  quality  of  the  Air  Force  medical  service.  The  objective  of  this 
research  effort  as  stated  by  the  Air  Force  Military  Personnel  Center  Surgeon  General  (AFMPC/SG) 
was  to  Increase  the  retention  rate  of  physician-officers  while  maintaining  the  standards  expected 
of  them.  This  objective  served  as  a  guide  in  developing  the  selection  model  discussed  in  this 
report.  AFMPC/SG  has  stated  that  an  improvement  in  the  ability  to  select  AFHPSP  candidates  could 

save  the  Air  Force  almost  $3  million  annually  (FY88  dollars)  through  a  reduction  of  1?  in  student 

attrition  rates. 

In  order  to  determine  what  characteristics  available  at  the  time  of  selection  would  best  meet 
the  objective  established  for  AFHPSP,  AFMPC/SG  had  suggested  that  a  data  base  be  developed  to 
track  the  progress  of  selectees  from  their  initial  medical  training  phases,  through  subsequent 
career  milestones,  to  the  time  they  leave  the  service.  Although  not  undertaken  as  part  of  the 

present  effort,  the  future  implementation  of  such  a  tracking  system  would  provide  feedback  to  the 
selection  committee,  who  could  then  make  improvements  to  the  AFHPSP  selection  process  ha-pd  on 
analysis  of  the  tracking  system  data. 
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III.  BACKGROUND 


Literature  Review 


A  literature  review  was  conducted  to  assess  the  existing  body  of  medical  school  selection 
research.  The  bulk  of  the  literature  addressed  the  relationships  among  individual  variables 
(cognitive,  demographic,  and  personality),  outcomes  such  as  medical  school  success  (course 
grades,  clinical  clerkship  ratings.  National  Board  of  Medical  Examiners'  tests),  and  medical 
specialty  preferences. 

In  general,  it  was  found  that  cognitive  indicators  such  as  undergraduate  grade  point  average 
(GPA)  and  scores  on  the  Medical  Curriculum  Aptitude  Test  (MCAT)  were  predictive  of  success  during 
the  academic  portion  of  medical  school,  but  not  predictive  of  success  during  the  clinical  portion 
(Carline,  Cullen,  Scott,  Shannon,  t  Schaad,  1983;  Kupfer,  Drew,  Curtis,  A  Rubinstein,  1  978).  This 
finding  led  to  the  attempt  to  identify  other  indicators  which  would  better  predict  clinical 
performance.  Although  a  number  of  studies  found  that  varying  combinations  of  demographic  and 
personality  factors  were  statistically  significart  in  determining  clinical  success  in  medical 
school,  no  real  consensus  existed  as  to  which  specific  noncognitive  factors  could  effectively 
predict  clinical  performance. 

The  implications  of  these  findings  for  the  improvement  of  the  AFHPSP  selection  procedures 
were  twofold.  First,  the  prominent  role  of  cognitive  factors  in  the  AFHPSP  selection  process 
duplicated  academic  screening,  which  medical  schools  already  do,  and  added  little  to  determining 
if  a  person  has  the  ability  to  become  a  successful  career  officer  and  physician.  Second,  the  Air 
Force  needed  to  explore  the  further  use  of  noncognitive  indicators,  such  as  family  background, 
not  only  to  improve  prediction  of  clinical  success,  but  also  to  improve  the  ability  of  the  Air 
Force  to  predict  the  success  of  the  individual  as  a  physician  and  as  an  officer,  and  the  length 
of  time  that  an  Individual  would  spend  in  the  service. 


Selection  Board  Composition  and  Procedures 


Before  discussing  the  evaluation  of  the  selection  board  procedures,  it  is  important  to  first 
describe  the  composition  and  the  operation  of  the  AFHPSP  selection  board.  The  selection  board  is 
composed  of  five  Medical  Corps  or  Medical  Service  Corps  officers  and  meets  approximately  12  times 
during  a  year's  selection  cycle  beginning  in  January  and  ending  in  mid-April.  Each  board  member 
individually  reviews  a  candidate's  application  and  scores  it  based  on  two  objective  measures 
(undergraduate  GPA  and  MCAT  scores)  and  four  subjectively  evaluated  factors  (work  experience, 
extracurricular  activities,  letters  of  recommendation,  and  military  career  orientation).  The 
board  members'  scores  for  each  of  the  six  selection  factors  are  then  added  together  to  form  a 
board  score  for  each  candidate.  The  candidates  are  then  rank-ordered  according  to  their  board 
scores,  and  the  selection  of  scholarship  recipients  is  made  based  upon  a  predetermined  cutoff 
score. 


Selection  Board  Procedures  Evaluation 


Analysis  of  selection  board  procedures  was  accomplished  by  using  the  results  from  the  1983 
selection  board  to  examine  two  issues:  (a)  the  level  of  interrater  consistency  across  the 
selection  board  judges  on  the  six  selection  factors,  and  (b)  the  ability  of  different  influencing 
factors  (which  included  the  six  selection  factors)  to  discriminate  between  those  individuals 
selected  and  those  not  selected. 
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Interrater  consistency  can  be  measured  by  two  intraclass  correlations  (Guilford,  1965).  One 
correlation,  R^,  is  a  measure  of  the  consistency  of  a  single  rater  as  compared  to  a 
hypothetically  similar  rater.  The  other  correlation,  R^,  is  a  measure  of  the  consistency  of 
group  judgment  for  all  of  the  raters  as  compared  to  a  hypothetically  similar  group  of  raters. 
(Both  of  the  correlations  range  in  value  from  0  to  1.  A  value  of  0  means  that  no  correlation 
exists,  whereas  a  correlation  of  1  is  interpreted  as  complete  agreement.) 

The  R^-|  and  R^  breakdowns  (based  on  five  board  members)  by  selection  factor  and  board 
week  were  computed  for  the  1983  selection  process.  A  correlation  of  .80  was  preestablished  as  a 

criterion  for  adequate  interrater  consistency.  Of  the  six  selection  factors,  only  GPA  and  MCAT 

(both  having  a  12  board  weeks'  average  R^  =  .88  and  R^  =  .97)  attained  this  level  of 
consistency.  These  two  factors  were  more  reliable  than  the  other  four  because  they  were 
quantitative  measures  and  disagreement  on  the  value  of  scores  for  these  two  factors  was  extremely 
rare.  The  other  four  factors  did  not  fare  well,  with  average  R-j -j  values  between  .14  and  .26 

and  average  R^  values  from  .43  to  .62.  These  findings  had  a  direct  impact  on  the  policy  model 

chosen,  as  discussed  in  the  next  section. 

The  second  issue  was  to  determine  which  factor  or  factors  could  be  said  to  "drive"  the 

selection  process.  GPA  and  MCAT  were  obvious  candidates  since  they  were  the  only  reliable 
factors  among  the  original  six.  In  addition  to  GPA  and  MCAT  scores,  another  influencing  factor, 
scholarship  fill,  was  uncovered  in  researching  the  board's  selection  process. 

Scholarship  fill  is  the  percentage  of  scholarships  awarded  out  of  the  total  scholarships 

available.  A  problem  which  surfaced  was  that  during  the  12-week  selection  period,  a 

progressively  smaller  percentage  of  scholarship  applicants  were  selected  as  the  AFHPSP  selection 
process  continued.  This  was  attributed  to  the  cutoff  scores  being  raised  as  the  selection 
process  continued  throughout  the  year.  As  a  result,  an  individual  selected  in  the  early  part  of 
the  selection  cycle  might  not  have  been  selected  had  that  applicant  met  the  board  in  the  11th  or 
12th  week.  The  sensitivity  of  selection  to  scholarship  fill  can  be  seen  in  the  FY83  selection 
board  results.  In  the  first  week,  the  cutoff  score  was  475  (out  of  a  possible  500)  and  the 
selection  rate  was  74%.  By  week  5,  the  cutoff  score  had  risen  to  480  and  the  selection  rate  had 
dropped  to  22%.  The  problem  has  since  been  alleviated  by  the  introduction  of  an 

AFHRL -recommended  constant  cutoff  score.  Since  the  introduction  of  this  improvement,  the  cutoff 
score  for  selection  into  AFHPSP  has  been  established  as  480  out  of  a  possible  500  points. 

The  three  factors — GPA,  MCAT,  and  scholarship  fill— were  evaluated  using  a  stepwise 
discriminant  procedure  (Nie,  Hull,  Jenkins,  Steinbrenner,  S  Bent,  1  975)  in  order  to  determine 
their  relative  contributions  to  AFHPSP  select/nonselect  outcomes.  All  three  of  the  factors  were 
significant  at  the  p  =  .0001  level.  Scholarship  fill  ranked  highest  in  contribution  to  the 
selection  decision  (with  a  standardized  coefficient  of  0.8276),  followed  by  MCAT  (0.6133),  and 
GPA  (0.2846),  respectively  (Martin,  1984). 


IV.  AFHPS.  POLICY  MODEL  DEVELOPMENT 


Approach 


In  order  to  assist  the  board  in  selecting  AFHPSP  recipients,  a  selection  policy  model  was 
needed  that  would  incorporate  selected  individual  attributes  along  with  board  members'  inputs. 
Ward  (1977)  discussed  two  techniques  available  for  individual  or  group  policy  development: 
judgment  analysis  (policy  capturing)  and  policy  specifying.  Policy  capturing  is  appropriate  when 
the  predictor  variables  are  quantifiable  and  the  within-group  correlation  (R^)  is  high  (above 
.80  in  this  case)  for  a  majority  of  the  variables.  Policy  specifying  requires  predictor 
variables  also  but  is  more  flexible  in  structure  than  policy  capturing  and  allows  for  the 
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development  and  specification  of  variables,  model  structure,  and  parameters.  Weights  are 
determined  “by  stating  desired  properties  of  and  relations  among  the  predicted  values  in 
sufficient  detail  that  the  numerical  weights  become  known"  (Ward,  1977).  Due  to  low  interrater 
agreement  across  several  of  the  variables  and  the  use  by  the  board  of  difficul  t-to-quantify 
information  such  as  letters  of  recommendation  and  work  experience,  the  policy-specifying 
technique  was  selected  as  the  primao'  research  methodology. 

The  four  steps  of  the  hierarchical  policy-specifying  process  have  been  defined  by  Ward,  Pina, 
Fast,  and  Roberts  (1979)  as  follows: 

1.  Select  one  or  more  experts  who  will  specify  the  policy  in  a  mathematical  form. 

2.  Identify  the  dependent  and  independent  variables.  Implicit  in  this  step  is  the 
identification  of  organizational  goals  and  constraints,  as  well  as  a  thorough  review  of  available 
data  associated  with  the  organizational  goals  (Hendrix,  Ward,  Pina,  &  Haney,  1979). 

3.  Combine  the  independent  variables  into  a  hierarchy  of  logical  pairs.  Combine  the 
functional  results  of  these  pairs  into  logical  pairs  until  a  pyramidal  hierarchy  is  defined. 

4.  For  each  pair  of  variables  and/or  functions,  assign  scores  or  "payoffs"  at  selected  value 
combinations  over  the  range  of  possible  values.  Then,  determine  a  mathematical  model 
representing  the  pairwise  hierarchical  combination.  After  the  completion  of  step  4,  the  policy 
model  (decision  making  process)  has  been  effectively  specified. 


Policy  Model  Formulation 


In  1983,  a  working  group  was  formed  consisting  of  representatives  from  AFMPC/SG,  Air  Force 
Recruiting  Service  (AFRS),  and  the  Air  Force  Human  Resources  Laboratory  (AFHRL).  This  group 
developed  a  top-down  structural  model  that  was  used  to  translate  organizational  goals  into  a 
hierarchy  of  logical  pairs  of  variables  and  functions.  The  approach  used  here  began  at  the  "top" 
by  identifying  "AFHPSP  Potential"  as  the  final  selection  measure  and  then  constructing  a 
hierarchy  below  it.  This  allowed  the  working  group  to  have  complete  control  over  the  development 
of  the  structure  of  the  policy  model.  Logical  combinations  of  pairs  of  variables  and  functions 
were  then  assigned  payoff  values.  The  developed  hierarchy  expressed  "AFHPSP  Potential"  as  a 
function  of  physician  and  officer  potentials  reflecting  the  policy  of  selecting  individuals  with 
promise  as  career-oriented  Air  Force  officers  and  as  physicians.  The  hierarchy  for  the  first 
policy  model  contained  36  input  variables  and  35  functions. 

Data  Collection 


The  large  number  of  variables  (36)  developed  for  the  policy  model  posed  two  significant  data 
collection  problems.  One  problem  was  that  the  information  required  by  the  policy  model  on  the 
applicant  was  more  extensive  than  that  being  collected  by  Air  Force  Recruiting  Service  at  the 
time.  In  order  to  collect  the  needed  data,  a  questionnaire  was  developed  by  AFHRL  with  input 
from  the  AFHPSP  working  group.  The  questionnaire  was  also  designed  to  collect  information  that 
AFMPC/SG  wanted  to  use  in  a  later  study  in  predicting  success  in  medical  school.  (See  Appendix  A 
for  further  discussion  and  a  copy  of  the  questionnaire.)  A  second  problem  was  that  of  missing 
data.  For  example,  some  data  required  by  the  policy  model  about  the  different  medical  schools 
accepting  applicants  were  not  available  for  use  or  were  available  for  only  a  limited  number  of 
schools.  Student-teacher  ratios  and  the  number  of  medical  school  graduates  able  to  get  their 
first  job  choice  upon  graduation  from  medical  school  are  examples  of  the  type  of  information  that 
was  missing. 
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Revised  Models 


Upon  completion  of  the  first  policy  model  by  the  working  group,  analysts  from  AFHRL  reviewed 
the  model.  This  review,  as  well  as  the  lack  of  available  data  for  the  model,  resulted  in  a 
series  of  revisions  that  made  the  model  more  tractable.  (In  making  any  revisions,  the  structure 
of  the  original  policy  model  was  paralleled.)  A  number  of  variables  were  dropped  due  to  data 
Insufficiency.  Additional  revisions  were  made  to  eliminate  redundancies.  For  example,  the 
amount  of  time  that  the  candidate  spent  working  with  patients  had  been  broken  into  volunteer  time 
and  compensated  time.  In  the  later  versions  of  the  model,  these  two  variables  were  combined  into 
one  variable  that  measured  the  amount  of  time  spent  working  with  patients. 

To  ensure  that  the  revised  policy  model  retained  an  adequate  degree  of  strength,  a 
correlation  was  calculated  between  payoffs  from  the  original  policy  model  and  the  final  policy 
model  using  the  data  collected  by  questionnaires  administered  to  students  at  the  Health 
Professions  Officer  Indoctrination  Course  (HPOIC).  Questionnaires  were  completed  by  264  AFHPSP 
scholarship  recipients  in  HPOIC.  AFHPSP  potential  scores  for  each  applicant  were  calculated  by 
both  models  and  then  rank-ordered.  The  correlation  calculated  between  the  two  policy  models  was 
.78.  Based  on  this  level  of  correlation,  the  working  group  agreed  that  the  two  policy  models 
were  not  substantially  different.  Figure  1  shows  the  hierarchy  used  for  the  final  policy  model. 
Table  1  presents  the  variables  included  in  the  model,  and  Table  2  shows  the  functions  that  relate 
the  variables  to  one  another.  The  mathematical  relationships  between  the  pairs  of  variables 
and/or  functions  are  shown  in  Appendix  B. 


AFHPSP 

POTENTIAL 


POTENTIAL 


IMS)  (HRS/WEEK) 


NOTE:  Input  variable*  ere  underlined. 


Figure  1.  Final  Policy  Model  Hierarchy. 
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Table  1.  AFHPSP  Variables 


Variable 

Range 

Mane 

Description 

XI 

200-400 

GPA 

Applicant's  undergraduate  GPA. 

X2 

0-80 

MCAT 

Applicant's  overall  MCAT  score. 

X3 

0-45 

MCAT 

Science 

Combination  of  the  individual's  sub¬ 
scores  for  Physics,  Biology,  and 
Chemistry  to  produce  an  “MCAT  Science" 
score. 

X4 

0-1 

Medical 

Family 

Indication  as  to  whether  either  parent 
of  the  applicant  is  a  physician. 

X5 

0-48 

D1  rect 

Patient 

Care  (mos) 

Number  of  months  that  the  applicant  has 
spent  caring  for  patients. 

X6 

0-20 

Dliect 

Patient 

Care  (hrs/wk) 

Average  number  of  hours  spent  weekly 
in  direct  patient  care 

X7 

0-20 

Distinctions 

Any  community,  athletic,  or  academic 
distinctions  the  person  might  have. 
Includes  any  offices  other  than 
president  or  captain  that  were  held 

In  civic  or  coamunity  clubs. 

X8 

0-10 

Memberships 

Number  of  community  or  athletic  member¬ 
ships  to  which  the  person  belongs. 

X9 

0-1 

Military 

Academy 

Completion 

Completion  of  one  of  the  three  U.S. 
military  academies. 

XI 0 

0-1 

ROTC 

Completion 

Completion  of  an  ROTC  program. 

XI 1 

0-1 

Prior 

Service 

Indication  of  prior  military  service. 

XI  2 

0-1 

Mil  itary 

Family 

Indication  as  to  whether  the  applicant 
has  anyone  of  his/her  immediate 
family  who  is  on  active  duty  or  who 
has  retired  from  active  duty. 

XI  3 

0-4 

Leadership 

Roles 

Number  of  organizations  or  teams  in 
which  the  person  was  a  president  or 
captai n. 

Note.  For  most  of  the  variables,  the  range  refers  to  the  range  of  values  found  in  the  data 
collected  (such  as  Variable  X7,  Distinctions)  or  a  range  that  is  implicit  in  the  variable  itself 
(such  as  Variable  X2,  MCAT  scores).  Those  variables  with  a  range  of  0-1  are  binary  variables 
that  correspond  to  a  yes/no  question.  For  example,  Variable  XT  1  asks  if  a  person  has  had  any 
prior  military  duty,  and  a  "0“  means  "yes"  and  a  "1"  means  “no." 
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Table  2.  AFHPSP  Functions 


Function 

Name 

Functional  Inputs 

FI 

Military  Affinity 

Variables  11  and  12 

F2 

Military  Education 

Variables  9  and  10 

F3 

Military  Experience 

Functions  1  and  2 

F4 

Specialty  Potential 

Variables  3  and  4 

F  5 

Service  Track 

Variables  7  and  8 

F6 

Care  Track 

Variables  5  and  6 

F7 

Achievement  Potential 

Functions  5  and  6 

F8 

Academic  Potential 

Variables  1  and  2 

F9 

Physician  Potential 

Functions  7  and  8 

F10 

Medical  Potential 

Functions  4  and  9 

FI  1 

Officer  Potential 

Function  3  and  Variable  13 

F 12 

AFHPSP  Potential 

Functions  10  and  11 

V.  POLICY  MODEL  VALIDATION 

To  ensure  Its  utility,  the  model  must  be  validated  over  both  the  short  and  the  long  terms.  A 
short-term  validation  of  the  model  was  accomplished  by  comparing  the  rank -orderings  of  the 
applicants  based  on  their  AFHPSP  board  scores  and  the  policy  model  scores.  For  purposes  of 
short-term  validation.  It  Is  Important  to  realize  that  the  model  was  never  designed  to  simulate 
the  board.  Rather,  It  was  simply  meant  to  provide  a  more  complete  picture  of  the  AFHPSP 
applicants  that  would  aid  In  the  selection  process. 


Short-Term  Validation 


Hit  Table 

Validation  of  the  policy  model  was  accomplished  using  the  data  collected  during  FY87.  Over 
800  applications  were  received  and  reviewed  by  the  board.  Of  these  applicants,  approximately  500 
had  completed  the  questionnaire  correctly  and  their  data  were  used  In  the  calculation  of  the  hit 
table.  Table  3  shows  that  the  board  and  the  policy  model  agreed  In  select/non-select  decisions 
for  59.2%  of  the  applicants,  and  disagreed  on  40.8%  of  the  applicants. 


Table  3.  Board  versus  Policy  Model  Decisions  (N  *  484) 


Board  Selection 

Decision 

Yes 

No 

Yes 

29.4% 

20.4% 

Policy  Selection 
Decision 

No 

20.4% 

29.8% 

Regression  Analysis 

The  next  step  In  the  short-term  validation  was  to  identify  which  characteristics  received 
differential  weighting  from  the  board  and  the  policy  model  and  caused  the  40.8%  disagreement. 
Two  multiple  regression  models  were  developed.  The  first  regression  used  the  board  score  minus 
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the  policy  model  score  as  the  dependent  variable  and  the  13  policy  model  variables  as  predictors 
(N  *  484).  The  second  regression  also  used  the  difference  between  board  score  and  the  policy 
model  score  as  the  dependent  variable,  but  used  the  six  board  variables  as  Independent  variables 
(N  *  484).  The  results  of  the  two  regressions  are  shown  In  Tables  4  and  5,  respectively. 

Table  4.  Board  Minus  Policy  Model  Scores 
and  Policy  Model  Variables 


Variable 

Coefficient 

“t"  statistic 

CONSTANT 

385.624 

54.646* 

GPA 

.1775 

14.559* 

MCAT 

.1509 

1.732 

MCAT  Science 

-1.1636 

-7.297* 

Medical  Family 

Direct  Patient 

-3.6712 

-3.086* 

Care  (mos) 

Direct  Patient 

.0309 

1.265 

Care  (hrs/wk) 

.0547 

1.102 

Distinctions 

-.0926 

-1.295 

Memberships 

Military  Academy 

-.0790 

-.637 

Completion 

-6. 5970 

-3.734* 

ROTC  Completion 

-7.2561 

-2.465* 

Prior  Service 

-3.0567 

-1.866 

Military  Family 

-3.4038 

-4.716* 

Leadership  Roles 

-3.0834 

-9.340* 

N  =  484 

R2  =  0.4932 

♦significant  at  the 

.05  level. 

Table  5. 

Board  Minus  Policy  Model 
and  Board  Variables 

Scores 

Variable 

Coefficient 

"f 

statistic 

CONSTANT 

237.6590 

35.225* 

GPA 

1.0143 

16.844* 

MCAT 

-  .2808 

-3.450* 

Work  Experience 

.8800 

7.028* 

Extracurricular 
Letters  of 

.4753 

3.388* 

Recoirmendation 

.9835 

5.108* 

Career  Potential 

.7541 

7.238* 

N  =  484 

R2 

=  0.5300 

♦significant  at  the  .05  level. 


In  the  regression  of  the  dependent  variable  and  the  13  policy  model  variables,  only  7  of  the 
variables  had  significant  "t"  scores  at  the  p  =  .05  level.  Of  these  seven,  only  GPA  exhibited  a 
positive  relationship  with  the  dependent  variable.  This  relationship  apparently  existed  because 
the  AFHPSP  selection  board  placed  more  emphasis  on  GPA  than  did  the  policy  model.  The  opposite 
can  be  said  for  the  other  six  significant  variables.  MCAT  Science,  Military  Family,  Medical 
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Family,  ROTC  Completion,  Military  Academy  Completion,  and  Leadership  Roles  all  received  a  greater 
emphasis  from  the  policy  model  than  they  did  from  the  selection  board. 

In  the  regression  using  the  six  board  variables,  all  of  the  variables  had  significant  "t" 
statistics.  The  MCAT  score  was  the  only  variable  that  had  a  negative  coefficient,  indicating 
that  the  policy  model  placed  a  greater  emphasis  on  that  variable  than  did  the  selection  board. 
Correspondingly,  variables  with  positive  coefficients  Indicated  that  the  selection  board  placed  a 
greater  emphasis  on  those  variables  than  did  the  policy  model. 

These  results  Indicate  that  the  primary  driver  of  the  disagreement  was  the  Information 
differences  that  existed  between  the  two  selection  systems  (the  policy  model  and  the  board).  The 
variables  from  the  policy  model  (except  for  MCAT)  that  were  not  statistically  significant  were  of 
little  value  In  distinguishing  between  policy  model  selectees  who  were  not  selected  by  the  board 
and  board  selectees  who  were  not  selected  by  the  policy  model;  i.e.,  these  variables  were 
considered  equally  or  not  at  all  by  the  two  selection  systems.  A  further  elaboration  of 
differences  and  similarities  between  the  selection  systems  may  be  found  In  Appendix  C,  which 
shows  how  policy  model  and  board  variables  relate  to  corresponding  (and  cross-applied)  selection 
score  outcomes.  That  the  MCAT  variable  was  statistically  insignificant  In  the  policy  model  could 
very  well  be  due  to  the  Influence  of  the  MCAT  Science  score  with  which  It  was  highly  correlated. 
Although  it  was  possible  to  identify  which  variables  received  the  greater  emphasis  (either  by  the 
selection  board  or  the  policy  model),  the  ability  to  state  precisely  which  variables  were  the 
most  useful  In  predicting  the  ability  of  a  person  to  become  a  career  officer-physician  In  the  Air 
Force  was  hampered  by  the  lack  of  a  long-term  data  base.  Despite  this  limitation,  the  working 
group  believed  that  the  policy  model  provided  a  usable  decision  aid,  and  a  decision  was  made  to 
implement  the  results  of  the  policy  model  starting  with  the  FY88  selection  boards. 


Long-Term  Validation 

Long-term  validation  requires  the  establishment  of  a  data  base  tracking  system  that  would 
follow  an  AFHPSP  selectee  throughout  his/her  career  as  an  officer-physician.  This  collection  of 
data  such  as  the  length  of  service,  assignments,  rate  of  promotion,  etc.  could  be  used  to  develop 
an  indicator  of  how  a  person's  career  has  progressed.  This  career  Indicator  would  then  reflect 
how  well  the  AFHPSP  board  made  its  selection.  An  analyst  could  maintain  the  selection  Input 
variables  by  cohort  (year  group)  and  gradually  add  performance  data  from  medical  school, 
residency  program,  and  Air  Force  active  duty  assignments. 

By  reviewing  the  input  variables  of  successful  officer-physicians  who  were  former  AFHPSP 
candidates,  a  determination  could  be  made  as  to  which  characteristics  (available  at  the  time  of 
selection)  best  predict  which  recipients  made  successful  career  officer-physicians.  This 
analysis  would  provide  insight  into  the  predictive  abilities  of  the  selection  variables  as  well 
as  a  basis  for  modifying  and  refining  the  policy. 


VI.  POLICY  MODEL  APPLICATIONS 

The  policy  model  was  designed  to  assist  or  complement  the  board  in  making  selections  of  the 
best  applicants  for  AFHPSP  scholarships.  This  objective  could  be  accomplished  by  using  the 
policy  model  in  one  or  more  of  the  following  ways: 

1.  Consistency  Check.  The  policy  model  could  be  used  to  sort  and  rank  individuals  according 
to  their  final  AFHPSP  potential  scores.  This  ranking  could  be  used  to  check  the  board's 
selections  for  consistency.  If  desired,  the  board  could  reconsider  nonselectees  in  the  top 
percentage  of  this  ranking  as  well  as  selectees  in  the  bottom  percentage  (top  and  bottom  cutoff 
percentages  to  be  determined). 
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2.  Additional  Board  Heater.  The  greater  variability  In  the  model's  scores  could  provide 

greater  dl serial nablllty  among  the  candidates  and  reduce  the  number  of  ties  (which  would  save 
time  since  It  Is  a  lengthy  process  to  rank-order  applicants  having  the  sane  board  score).  With 
six  scores  combined  to  arrive  at  the  board's  score,  rather  than  five,  the  cutoff  score  for 
selectees  and  non-selectees  would  be  adjusted  accordingly.  A  slightly  different  benefit  to  be 
derived  from  nodel  scores  as  a  sixth  board  nenber  would  be  as  an  aid  In  selecting  the  middle 
people.  Selection  board  neebers  tend  to  agree  very  easily  on  the  top  and  the  bottom  groups  of 
applicants  but  have  a  great  deal  of  difficulty  selecting  people  from  the  middle  ranges.  The 

policy  model  scores  could  be  added  to  this  middle  group  to  aid  In  the  selection  decision. 

3.  Applicant  Sort.  The  policy  model's  rank -ordered  applicant  listing  could  be  used  to 

screen  out  applicants  prior  to  the  start  of  the  selection  procedures.  Individuals  whose  payoff 
value  1$  less  than  some  specified  minimum  score  would  not  be  considered  for  scholarship 

selection.  This  approach  would  allow  board  members  more  time  to  examine  the  files  of  those 

applicants  who  are  competing  for  scholarships. 

4.  Tie-Breaker.  The  policy  model  could  be  used  to  aid  the  AFHPSP  board  In  making 
select/non'select  decisions  In  those  cases  where  board  members  assign  the  same  scores  to  two  or 
more  appl1:ants.  In  this  approach,  ties  could  be  broken  by  using  applicant  scores  based  on  a 
model  that  considers  factors  the  board  does  not  address. 


VII.  CONCLUSIONS/RECOMMENDATIONS 

• 

This  study  developed  a  methodology  and  a  data  collection  questionnaire  to  aid  in  the 
selection  of  AFHPSP  recipients.  A  comparison  between  the  computer-based  selection  system  and  the 
AFHPSP  selection  board  showed  disagreement  in  40.8S  of  the  select/non-select  decisions. 

Regression  analysis  Indicated  the  cause  of  the  disagreement  was  that  the  policy  model  and  the 

board  placed  different  emphasis  on  certain  variables  and  considered  different  types  and  numbers 

of  decision  variables.  Without  a  student/officer  career  tracking  system.  It  was  not  possible  to  _• 

make  an  objective  comparison  between  the  long-term  effectiveness  of  candidates  selected  by  either 

system. 

Although  this  lack  of  longitudinal  data  for  comparative  purposes  did  prevent  a  determination 
as  to  which  method  makes  the  best  selection  decision,  the  following  recommendations  can  be 
offered:  • 


1.  Continue  to  administer  the  data  collection  questionnal re  through  the  Air  Force  Recruiting 
Service.  This  will  allow  data  to  be  gathered  on  the  personal  characteristics  of  the  applicant 
pool.  These  data  will  serve  as  the  basis  for  a  long-term  data  base  and  for  future  research  to 
design  the  most  effective  selection  system  possible. 

2.  Establish  a  long-term  tracking  system  that  will  maintain  the  results  of  both  the  AFHPSP 
selection  board  and  the  AFHPSP  policy  model.  In  addition,  selected  career  Information  (as 
determined  by  AFMPt/SG)  would  be  entered  into  the  data  base  to  provide  career  information  on 
AFHPSP  scholarship  recipients. 

By  adopting  these  recommendations,  the  groundwork  will  be  laid  for  a  future  study  to 
demonstrate  the  most  effective  synthesis  of  both  the  AFHPSP  selection  board  process  and  the 
AFHPSP  policy  model.  Such  a  future  study  should  demonstrate  which  of  the  demographic 
characteristics  of  AFHPSP  applicants  are  most  closely  related  to  their  becoming  successful  career 
physician-officers. 

3.  It  Is  also  reconmended  that  the  AFHPSP  selection  board  use  the  results  from  the  policy 
model  as  a  tie-breaker  to  make  more  consistent  and  Informed  selections.  The  AFHPSP  policy  model 
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provides  consistent  applicant  evaluation  and  incorporates  Information  the  AFHPSP  selection  board 
does  not  directly  consider.  Use  of  the  AFHPSP  policy  model  will  be  a  significant  Improvement  in 
selecting  successful,  career-minded  officer-physicians. 
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APPENDIX  A:  AFHPSP  QUESTIONNAIRE 


The  AFHPSP  questionnaire  was  designed  by  AFHRL  with  Input  from  the  AFHPSP  working  group  and 
Is  currently  administered  by  the  Medical  Recruiting  Division,  Air  Force  Recruiting  Service 
(RS/RSHM).  A  questionnaire  and  corresponding  scan  sheet  are  completed  by  each  AFHPSP  applicant 
at  a  recruiting  office.  When  an  applicant  completes  his/her  application  package.  It  is  mailed  to 
RS/RSW  which  has  the  scan  sheet  read  with  an  optical  scanner.  Data  resulting  from  the  scanning 
process  are  then  transferred  to  the  Air  Force  Military  Personnel  Center's  Surgeon  General's 
office  which  compiles  the  data  for  Its  own  use.  Data  from  questions  1  through  99  are  used  to 
determine  the  values  for  the  Input  variables.  The  results  from  the  remaining  questions  are 
recorded  to  be  used  In  the  long-term  validation. 
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Unless  otherwise  specified,  the  time  frame  considered  when  completing  this  questionnaire  should 
be  from  your  freshman  year  In  high  school  through  today. 


Items  1  through  6.  If  you  received  a  distinction  listed  below,  mark  response  “A."  Otherwise 
leave  blank. 


1.  Eagle  Scout 

2.  Silver  Beaver  Award 

3.  Civic  activity,  honor,  or  award 

4.  Outstanding  Young  Man/Woman  of  America  Award 

5.  Organizer  of  a  college  political  group  or  campaign 

6.  American  Legion  Award 


Items  7  through  21.  If  you  were  Involved  In  any  of  the  following  community  service  activities 
for  at  least  1  year,  mark  response  "A,"  “B,"  or  “C"  below  to  Indicate  the  highest  position  you 
held  in  each  activity.  Leave  blank  If  you  did  not  participate  In  an  activity  for  at  least  1  year. 

A.  Member 

B.  Office  holder 

C.  President 

7.  Red  Cross 

8.  Political  groups 

9.  Big  Brother/Big  Sister 

10.  Student  government 

11.  Service  oriented  fraternltles/organlzatfons 

12.  Arnold  Air  Society 

13.  Elderly  programs 

14.  Boy /Girl  Scouts 

15.  Youth  athletics 

16.  YMCA,  YWCA  Volunteer  Work 

17.  Religious  group 

18.  United  Way 

19.  4-H  Club 

20.  Choir 

21.  Other  (Please  specify  in  the  Conments  section.) 
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I  teas  22  through  39.  If  you  attained  an  achievement  listed  below,  mark  response  "A." 

Otherwise  leave  blank. 

i 

22.  Received  research  grant  * 

23.  Participated  In  an  undergraduate  research  program 

24.  Presented  scientific  paper 

25.  Received  project  award 

26.  Published  scientific  paper 

27.  Won  speech  or  debate  contest 

28.  Received  acting,  playwriting,  or  other  dramatic  award  £ 

29.  Published  poems,  stories,  essays,  or  articles 

30.  Served  as  editor  or  feature  writer  for  paper 

31.  Received  literary  award  or  prize 

32.  Received  musical  competition  award 

33.  Participated  In  a  public  recital 

34.  Received  art  competition  award  ® 

35.  Received  an  athletic  distinction 

36.  Received  an  outstanding  employee  award 

37.  Served  as  drill  team  or  cheer  leader  captain 

38.  Served  as  a  sports  manager 

39.  Other  (Please  specify  In  the  Comments  section.) 


Items  40  through  45.  When  answering  Items  40  through  45,  consider  only  those  periods  when  you 
were  a  full-time  student  (high  school  and  college). 

40.  Approximately  how  many  months  have  you  worked  In  a  paying  job(s)  requiring  direct  patient 
care? 


A. 

0-3 

D.  12-15 

G. 

24-27 

J.  36-39 

M.  48  or  more 

B. 

4-7 

E.  16-19 

H. 

28-31 

K.  40-43 

C. 

8-11 

F.  20-23 

I. 

32-35 

L.  44-47 

41.  On 

the  average,  how  many 

hours 

per  week 

have  you  worked  In  a  paying  job(s)  requiring  direct 

patient  care? 

A. 

0-1 

0,  6-7 

G. 

12-13 

J.  18-19 

B. 

2-3 

E.  8-9 

H. 

14-15 

K.  20  or  more 

C. 

4-5 

F.  10-11 

I. 

16-17 

42.  Approximately  how  many  months  have  you  done  volunteer  work 

requiring  direct  patient  care? 

A. 

0-3 

D.  12-15 

G. 

24-27 

J.  36-39 

M.  48  or  more 

B. 

4-7 

E.  16-19 

H. 

28-31 

K.  40-43 

C. 

8-11 

F.  20-23 

I. 

32-35 

L.  44-47 

43.  On  the  average,  how  many  hours  per  week  have  you  done  volunteer  work  requiring  direct 
patient  care? 

A.  0  0.  3  G.  6  J.  9 

B.  1  E.  4  H.  7  K.  10  or  more 

C.  2  F.  5  I.  8 
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44.  Approximately  how  many  months  have  you  worked  In  a  job(s)  not  requiring  direct  patient  care? 


A.  0-3  0. 

12-15 

G. 

24-27 

J. 

36-39  M. 

48  or  more 

B.  4-7  £. 

16-19 

H. 

28-31 

K. 

40-43 

C.  8-11  F. 

20-23 

I. 

32-35 

L. 

44-47 

45.  On  the  average, 
patient  care? 

how  many 

hours 

per  week 

have 

you  worked  In 

a  job(s)  not  requiring  direct 

A.  0-1  0. 

6-7 

G. 

12-13 

J. 

18-19 

B.  2-3  E. 

8-9 

H. 

14-15 

K. 

20  or  more 

C.  4-5  F. 

10-11 

I. 

16-17 

Items  46  through  51.  When  answering  Items  46  through  51.  consider  only  those  periods  when  you 
were  not  a  full-time  student. 

46.  Approximately  how  many  months  have  you  worked  In  a  paying  job  requiring  direct  patient  care? 


A. 

0-3  D. 

12-15 

G.  24-27 

J.  36-39 

M.  48  or  more 

B. 

4-7  E. 

16-19 

H.  28-31 

K.  40-43 

C. 

8-11  F. 

20-23 

I.  32-35 

L.  44-47 

47. 

On 

the  average. 

how  many 

hours  per  week 

have  you  worked 

In  a  paying  job(s)  requiring  direct 

patient  care? 

A. 

0-1  0. 

6-7 

G.  12-13 

J.  18-19 

B. 

2-3  E. 

8-9 

H.  14-15 

K.  20  or  more 

C. 

4-5  F. 

10-11 

I.  16-17 

48. 

Approximately  how  many  months  have  you  done  volunteer  work  requiring  direct  patient  care? 

A. 

0-3  D. 

12-15 

G.  24-27 

J.  36-39 

M.  48  or  more 

B. 

4-7  E. 

16-19 

H.  28-31 

K.  40-43 

C. 

8-11  F. 

20-23 

I.  32-35 

L.  44-47 

49. 

On 

the  average. 

how  many 

hours  per  week  have  you  done  volunteer  work  requiring  direct 

patient  care? 

A. 

0  0.  3 

G. 

6 

J.  9 

B. 

1  E.  4 

H. 

7 

K.  10 

or  more 

C. 

2  F.  5 

I. 

8 

50. 

Approximately  how  many  months  have  you  worked  In  a  job(s) 

not  requiring  direct  patient  care? 

A. 

0-3  D. 

12-15 

G.  24-27 

J.  36-39 

M.  48  or  more 

B. 

4-7  E. 

16-19 

H.  28-31 

K.  40-43 

C. 

8-11  F. 

20-23 

I.  32-35 

L.  44-47 
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51.  On  the  average,  how  many  hours  per  week  have  you  worked  In  a  paying  job(s)  not  requiring 
direct  patient  care? 


A. 

0-1 

D. 

6-7 

G. 

12-13 

J. 

18-19 

B. 

2-3 

E. 

8-9 

H. 

14-15 

K. 

20  or  mere 

C. 

4-5 

F. 

10-11 

I. 

16-17 

52.  Are  either  of  your  parents  physicians? 

A.  Yes 

B.  No 


Items  53  through  65.  If  you  were  Involved  In  any  of  the  following  activities  for  at  least  1 
year,  mark  response  "A,"  "B,“  or  "C"  below  to  indicate  the  highest  position  you  have  held  In  each 
activity.  Leave  blank  If  you  did  not  participate  In  an  activity  for  at  least  1  year. 

A.  Member 

B.  Office  holder 

C.  President 

53.  College  debating  team 

54.  Dramatic  group 

55.  Social  fraternity /sorority 

56.  Honor  or  recognition  society 

57.  Departmental  club 

58.  University  Recreation  Association/Organization 

59.  Theatrical  groups 

60.  College/comnunlty  band  or  orchestra 

61.  National  Students  Association 

62.  Junior  Achievement 

63.  Radio  or  TV  station 

64.  Professional  organization 

65.  Other  (Please  specify  In  the  Comments  section.) 


Items  66  through  70.  If  you  have  participated  three  times  per  week  for  at  least  one  year  in 
any  of  the  following  physical  fitness  methods  then  mark  response  "A.”  Otherwise  leave  blank. 

66.  Height  lifting 

67.  Jogging 

68.  Swlmning 

69.  Aerobics 

70.  Other  (Please  specify  In  the  Conments  section.) 


Items  71  through  85.  Several  college  Intramural  sports  you  may  have  participated  in  are  listed 
below.  If  you  participated  in  a  sport  for  at  least  two  seasons  mark  response  "A. “  Otherwise 
leave  blank. 


71.  Football 

79. 

Swi  inning 

72.  Basketball 

80. 

Skiing 

73.  Baseball 

81. 

Wrestl  ing 

74.  Track 

82. 

Crew 

75.  Hockey 

83. 

Soccer 

76.  Golf 

84. 

Fencing 

77.  Tennis 

85. 

Other  (Please  specify  in  the  Comnents 

78.  Volleyball 

section. ) 

Items  86  through  100.  If  you  were  involved  in  any  of  the  following  college  varsity  sports  for 
at  least  one  season,  mark  response  "A,"  "B,"  or  “C“  below  to  indicate  highest  level  of 
achievement.  Lease  blank  if  you  did  not  participate  in  a  varsity  sport  for  at  least  one  season. 


A.  Member 

B.  Letter 

C.  Captain 

86. 

Footbal 1 

94. 

Swinming 

87. 

Basketball 

95. 

Skiing 

88. 

Basebal 1 

96. 

Wrestl  ing 

89. 

Track 

97. 

Crew 

90. 

Hockey 

98. 

Soccer 

91. 

Golf 

99. 

Fencing 

92. 

Tennis 

100. 

Other  (Please  specify  in  the  Comnents 

93. 

Cross  Country 

section. ) 

101.  Have  you  had  any  prior  military  experience? 

A.  Yes,  as  an  officer 

B.  Yes,  enlisted 

C.  Yes,  both  enlisted  and  as  an  officer 

D.  No,  not  appl icable 

102.  If  you  responded  "A,”  "B,"  or  "C"  to  question  #101,  then  Indicate  the  source  of  your 
experience. 

A.  Active  duty 

B.  Reserves 

C.  National  Guard 

D.  Active  duty  and  Reserves 

E.  Active  duty  and  National  Guard 

F.  Reserves  and  National  Guard 

G.  Other  (Please  specify  in  the  Conments  section.) 


103.  If  applicable.  Indicate  your  source  of  coamlssionlng. 

A.  Academy 

B.  ROTC 

C.  OTS 

0.  Other  (Please  specify  In  the  Conments  section.) 

E.  Mot  applicable 

104.  If  you  are  currently  enrolled  In  a  military  training  program,  mark  the  appropriate 
response.  Otherwise  leave  blank. 

A.  College  ROTC 

B.  Air  Force  Academy 

C.  West  Point 

D.  Naval  Academy 

E.  Other  (Please  specify  In  the  Conments  section.) 


Items  105  through  110.  If  any  of  your  relatives  listed  below  are  either  retired  from  20  years 
or  more  of  active  military  service  or  presently  on  active  duty  status  in  the  Armed  Forces,  then 
mark  response  “A. “  Otherwise  leave  blank. 

105.  Mother 

106.  Father 

107.  Brother 

108.  Sister 

109.  Aunt 

110.  Uncle 


111.  Old  either  of  your  parents  serve  on  active  duty  at  all  during  your  lifetime? 

A.  Yes 

B.  No 

112.  Approximately,  how  many  people  live  In  or  near  the  area  ( i . e. ,  city,  town,  farm,  etc.) 
where  you  spent  the  most  time  when  you  were  growing  up? 


A.  1  -  1,000 

0. 

50,001 

-  200,000 

G.  1,000,001  or  more 

B.  1,001  -  10,000 

E. 

200,001 

-  500,000 

C.  10,001  -  50,000 

F. 

500,001 

-  1,000,000 

113.  Do  you  have  a  close  family  friend  who  Is  a  physician? 

A.  Yes 

B.  No 
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114.  Which  studies  are  more  Interesting  to  you? 

A.  Science 

B.  Humanities 

C.  Other  (Please  specify  In  the  Comments  section.) 

115.  Which  studies  are  easier  academically? 

A.  Science 
8.  Humanities 

C.  Other  (Please  specify  In  the  Comments  section.) 


Items  116  through  118.  Use  the  options  provided  below  to  answer  the  following  three  questions. 


A.  Family  Physician 

I. 

Obstetrician  /Gynecologist 

B.  Aerospace  Medical  Physician 

J. 

Pathologist 

C.  Pediatrician 

K. 

Diagnostic  Radiologist 

D.  Internist 

L. 

Anesthesiologist 

E.  Surgeon 

M. 

Neurologist 

F.  Urologist 

N. 

Psychiatrist 

G.  Opthalmologlst 

H.  Orthopedic  Surgeon 

0. 

No  Applicable/Other  (Please  specify 
In  the  Comments  section.) 

116.  If  you  ever  had  any  major  Injury  or  Illness,  Indicate  the  type  of  specialist  who  treated 
you  for  It. 

117.  If  a  member  of  your  Immediate  family  had  a  major  Illness  requiring  extended  contact  with 
physicians  or  hospitals.  Indicate  the  type  of  specialist  who  treated  him/her  for  It. 


118.  If  you  had  to  choose  a  medical  specialty  now,  which  one  would  you  choose? 


APPENDIX  B:  POLICY  MODEL  MATHEMATICAL  RELATIONSHIPS 


This  appendix  contains  the  functional  relationships  (In  terms  of  variables  and  variable 
names),  the  mathematical  formulas  for  the  pairs  of  variables  and/or  functions  and  their  payoff 
tables.  The  mathematical  relationships  established  for  the  pairs  of  variables  and/or  functions 
were  derived  through  the  use  of  expert  judges.  The  AFHPSP  working  group  provided  the  expertise 
needed  to  develop  the  relationships.  In  each  relationship  the  experts  determined  the  values  for 
each  of  the  four  corner  points  for  each  pair  of  variables  as  well  as  Interactions  between  the 
variables.  This  Information  was  used  In  a  computer  program  to  derive  the  mathematical  equations 
shown  In  this  appendix.  Experts  then  reviewed  each  of  the  payoff  tables  to  ensure  that  they  were 
representative  of  the  desired  Interactions  among  the  pairs  of  variables  and/or  functions. 
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APPENDIX  C:  REGRESSION  EQUATIONS  FOR  PREDICTING 
POLICY  MODEL  AND  BOARD  SCORES 


This  appendix  presents  the  results  of  four  regressions.  Table  C-l  contains  the  results  of 
the  policy  model  variables  regressed  against  the  policy  model  scores  and  board  scores,  while 
Table  C-2  presents  the  results  of  the  board  variables  regressed  against  both  scores. 

The  R1’  values  for  the  regressions  of  the  policy  model  variables  on  policy  model  scores  and 
the  board  variables  on  the  board  scores  (0.979  and  0.895  respectively)  show  that  each  set  of 
variables  explain  a  large  proportion  of  the  variance  in  both  the  policy  model  and  board  scores. 
Lower  R2's  for  the  other  two  regressions  (policy  model  variables  -  board  scores,  0.408,  and 
board  variables  -  policy  model  scores,  0.378)  Indicate  that  although  the  information  used  by  the 
policy  model  or  the  board  could  explain  a  proportion  of  the  ratings'  variance  from  the  other 
selection  method,  they  primarily  point  to  the  fact  that  each  method  was  designed  for  different 
purposes  and  considered  different  Information. 
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Table  C-l.  Relationship  of  Policy  Model  Variables 
to  Policy  Model  and  Board  Scores 


Policy  model  scores 

Board 

scores 

Variable 

Coefficient 

t-statlstlc 

Coefficient 

t-statlstlc 

CONSTANT 

-0. 764 

-0.668 

384.860 

55.106* 

6PA 

0.003 

1.445 

0.180 

14.947* 

MCAT 

0.034 

2.428* 

0.185 

2.147* 

MCAT  Science 

1.567 

60.640* 

0.403 

2.556* 

Medical  Family 

3.369 

17.478* 

-0.302 

-0.257 

Direct  Patient 

Care  (mos) 

0.018 

4.676* 

0.049 

2.044* 

Direct  Patient 

Care  (hrs/wk) 

0.021 

2.560* 

0.075 

1.534 

Distinctions 

0.133 

11.467* 

0.040 

0.568 

Memberships 

0.070 

3.493* 

-0.009 

-0.072 

Military  Academy 
Completion 

8.610 

30.078* 

2.013 

1.151 

ROTC  Completion 

9.864 

20.679* 

2.607 

0.895 

Prior  Service 

6.942 

26.163* 

3.886 

2.397* 

Military  Family 

5.616 

48.030* 

2.213 

3.098* 

Leadership  Roles 

3.080 

57.579* 

-0.003 

-0.010 

N  =  484 

R2 

=  0.979 

R2 

=  0.408 

♦significant  at  the 

.05  level. 

Table  C-2.  Relationship  of  Board  Variables 


to  Policy  Model  and  Board  Scores 

Policy  model  scores 

Board 

scores 

Variable 

Coefficient  t-statlstlc 

Coefficient 

t-statlstlc 

CONSTANT 

-14.871  -2.409* 

222. 788 

76. 323* 

GPA 

0.007  0.125 

1.021 

39.193* 

MCAT 

1.236  16.600* 

0.955 

27.121* 

Work  Experience 

0.132  1.151 

1.012 

18.675* 

Extracurricular 

0.355  2.768* 

0.830 

13.681* 

Letters  of 

Recommendation 

-0.223  -1.265 

0.761 

9.131* 

Career  Potential 

0.1  90  1.993* 

0.944 

20.941* 

N  =  484 

R2  =  0.979 

R2  = 

0.408 

♦significant  at  the  .05  level. 
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